Various electronic mechanisms are used for storing data which multiple parties need to access, modify and/or maintain, including electronic ledgers and database managements systems.
A ledger may be a collection of entries (obligations, assertions, debts, credits, etc.) in a notebook or other physical or electronic form and are akin to a transaction log whereby the current “state” of a ledger may be ascertained by netting or otherwise totaling all of the entries up to the current time period. For example, “Party A loans $X to Party B” could be an entry representative of a transaction in a ledger. “Party B repays $X to Party A” may be a subsequent entry of another transaction in that ledger. The net result of these two entries is the extinguishing of the debt of B to A. Ledgers typically utilize double-entry book keeping whereby separate ledger entries, or separate ledgers, are maintained for each side (account/party) to a transaction and transactions are recorded as a pair of opposing transactions, e.g. credits vs. debits, to each respective account/party, either in the same ledger or in separate ledgers, each maintained by the respective party.
Ledgers may be held by individual parties, or ledgers may contain entries for multiple parties and be replicated/distributed amongst a variety of sources. A ledger which comprises many distributed copies may referred to as a replicated ledger. An example of an electronic replicated ledger is the “blockchain” methodology employed by the bitcoin digital currency. Generally, a block chain, or blockchain, is a distributed database that maintains a continuously-growing list of data records, typically hardened against tampering and revision. It consists of data structure blocks which, in some implementations, hold exclusively data and both data and programs in other implementations, wherein each block stores batches of individual transactions and the results of any blockchain executables. Each block typically further contains a timestamp and information linking it to a previous block. Effectively, blockchain is an electronic public replicated ledger in which transactions, such as those involving the cryptographic currency bitcoin, are recorded. Each of the replicated blockchains communicates with the others via a network, such as the Internet. The Bitcoin blockchain operates completely transparently, so all data is transmitted to, and is readable by, all participants in the bitcoin system. That is, each party in the bitcoin system, with some exceptions, maintains a copy of the ledger, stored by their own copy of the blockchain, in which copies of all transactions are recorded, referred to as “full replication.” In the case of bitcoin, this replicated ledger makes all transactions “open transactions” and viewable by all participants on the blockchain network which is a necessary property required to prevent double spending of bitcoins, i.e., parties attempting to send the same bitcoin to multiple parties. This property of visibility of all transactions in the bitcoin network is also a drawback of a blockchain, because it does not allow for the confidentiality of transactions. Every participant in the bitcoin network has access to every transaction on the blockchain. This facilitates the ability to track digital assets, e.g. bitcoins. While the integrity of transactions recorded in each ledger is cryptographically protected, i.e. “signed,” via a transacting party's privately held cryptographic key, if someone were to steal a blockchain/bitcoin user's private key, the thief would have all of the information necessary, e.g. the transactional record and the cryptographic key thereto, to be able to see all of the transactions to which the user is a party, and the thief would be able to create transactions using the private key without the consent of the true owner of the private key.
Using the replicated ledgers of blockchain along with cryptographically linking/chaining the transactions stored therein enable all users to ensure the reliability of the transaction data, i.e. that transactions are recorded accurately and subsequent thereto, protected from alteration, as each user has a copy of all of the transactions and any unintended alterations to a transaction, e.g. via errors or fraudulent activity, are readily detectable via both the cryptographic discrepancies within the chained transactions that would be created as well as the discrepancies that such alterations will create among the various copies of the blockchain ledger.
A database is a structured collection of information or content, typically held in a computer, e.g. stored in a memory or other storage device that can be readily accessed, managed and updated, for storing the current value or net/cumulative result of a series of transactions. As used herein, a database refers not only to the underlying data structure(s) which are used to actually contain data but also the mechanisms coupled therewith to enable access, management, updating, etc. Together, this may also be referred to as a database management system (“DBMS”). As opposed to a ledger which records a sequence of transactions, a database typically records only the net result of the performance of those transactions. While the transactions recorded in ledgers are periodically netted to determine a current state, databases reflect the current state of data as soon as a transaction has been “committed,” i.e., the record in the database has been updated in manner considered to be permanent, e.g. visible to all users of that database.
Usually, the information is organized in a structured manner, i.e. using a particular format, protocol or structure for organizing and storing the data therein, and the information may be accessed, or edited via transactions, i.e. single database operations, according to a particular set of principles. In mission critical implementations where the reliability of the data stored in a database is desirable, databases may be implemented in accordance with certain properties which guarantee the reliable processing of transactions. The properties may include atomicity, consistency, isolation, and durability, commonly referred to as “ACID”. Adherence to these properties by a database/DBMS helps to guarantee that database transactions are processed reliably.
Atomicity requires that each transaction be “all or nothing”: if one part of the transaction fails, then the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed, i.e. completed, transaction appears (by its effects on the database) to be indivisible (“atomic”), and an aborted transaction does not happen.
The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules.
The isolation property ensures that the concurrent execution of transactions results in a system state that would be obtained if transactions were executed serially, i.e., one after the other. Providing isolation is the main goal of concurrency control. Depending on the concurrency control method (i.e., using strict, as opposed to relaxed, serializability), the effects of an incomplete transaction might not even be visible to another transaction.
The durability property ensures that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) may be recorded in a non-volatile memory.
However, the ACID approach to database management has drawbacks. For example, most implementations of the ACID approach require that data/records in the database be locked while that data is being interacted with, e.g. modified. This may effectively serialize access to data by multiple requestors and impede any operations which are dependent thereon.
Many business applications rely upon centralized databases/DBMS's, i.e. a database under the control of single or central entity, which, because they do not feature the replicated structure of blockchain or the cryptographic chaining of transactions, are typically implemented as a System of Record. A system of record (SOR) or Source System of Record (SSoR) is data management term for an information storage system, e.g. a computer implemented database/DBMS that is designated as the authoritative data source for a given data element or piece of information. Accordingly, while other entities may maintain copies of the data stored by an SOR, in the event of dispute between the value of particular data as stored in the SOR and as stored in a copy, the value stored by the SOR will be considered the true value. The need to identify systems of record can become acute in organizations where management information systems have been built by taking output data from multiple source systems, re-processing this data, and then re-presenting the result for a new business use. In these cases, multiple information systems may disagree about the same piece of information. These disagreements may stem from semantic differences, differences in opinion, use of different sources, differences in the timing of the extraction, transformation, and loading operations that create the data they report against, or may simply be the result of bugs. The integrity and validity of any data set is open to question when there is no traceable connection to a good source, such as a known System of Record. Where the integrity of the data is vital, if there is an agreed system of record, the data element must either be linked to, or extracted directly from it. Generally, a “system of record” approach may be used where there is a single authority over all data consumers, and those consumers have similar needs.
Generally a system of record (“SOR”) model is used for recording business related data such as transactions and agreements. In a SOR model, a trusted party holds and exclusively controls records of transactions in a centralized database. Individuals or other entities place their trust in the institution that hosts/controls the SOR, or otherwise agree that the SOR is the authoritative data source. Government and government agencies, financial institutions and even private/public companies may host/control the data and the SOR. For example, banks, 401k providers, utility companies, and many of the service agencies that people or business entities transact with are the SOR for that individual's or business entity's transaction records, e.g. account balance and/or transaction history with that service or agency. In the event of a dispute as to whether data in the SOR is correct as compared to another copy of that data which may differ, the institution that holds the SOR is typically deemed to have the correct data unless there is evidence to the contrary. Alternatively, when both parties are large institutions and neither institution is the SOR (e.g. two major financial institutions, such as two banks), messages are exchanged for every transaction and at the end of a period of time, e.g. at the end of the business day, a reconciliation process is undertaken by which each party validates their mutual understanding of each transaction which “seals” the transactions and, e.g. any end of day account balance resulting therefrom. In the event of a dispute, such as due to a bug, lost message or tampering, the parties must undertake a resolution process to determine the correct results, e.g. by reviewing network communication logs and/or transactional timestamps to determine the order of events. The SOR model, and the reconciliation process, referred to as a “trust and reconciliation” process, are commonly used in the implementation of electronic financial instrument trading systems.
Financial instrument trading systems are one example of complex systems that utilize databases according to an SOR model. Generally, a financial instrument trading system, such as a futures exchange, referred to herein also as an “Exchange”, such as the Chicago Mercantile Exchange Inc. (CME), provides a contract market where financial instruments, for example futures, options on futures and spread contracts, are traded among market participants, e.g. traders, brokers, etc. Futures is a term used to designate all contracts for the purchase or sale of financial instruments or physical commodities for future delivery or cash settlement, and which are traded on a commodity futures exchange. A futures contract is a standardized legally binding agreement to buy (long) or sell (short) a commodity or financial instrument at a specified price at a predetermined future time. An option is the right, but not the obligation, to sell (put) or buy (call) the underlying instrument (for example, a futures contract) at a specified price within a specified time. The commodity or instrument to be delivered in fulfillment of the contract, or alternatively the commodity, instrument or reference for which the cash market price shall determine the final settlement price of the futures contract, is known as the contract's “underlying” reference, instrument or commodity, also referred to as the “underlier.” The terms and conditions of each futures contract are standardized as to the specification of the contract's underlier, the quality and quantity of such underlier, delivery date, and means of contract settlement, i.e. physical delivery or cash settlement. Cash Settlement is a method of settling a futures contract whereby the parties effect final settlement when the contract expires by paying/receiving the pecuniary loss/gain of the contract, e.g. by comparing the contract price to the market price or other reference price of the underlier at the time of settlement, related to the contract in cash, rather than by effecting physical delivery, i.e. the actual exchange of the underlying reference or commodity at a price determined by the futures contract.
Typically, the Exchange provides for centralized “clearing” by which all trades are confirmed and matched, and open positions are settled each day until expired (such as in the case of an option), offset or delivered. Matching, which is a function typically performed by the Exchange, is a process, for a given order which specifies a desire to buy or sell a quantity of a particular instrument at a particular price, of seeking/identifying one or more wholly or partially, with respect to quantity, satisfying counter orders thereto, e.g. a sell counter to an order to buy, or vice versa, for the same instrument at the same, or sometimes better, price (but not necessarily the same quantity), which are then paired for execution to complete a trade between the respective market participants (via the Exchange) and at least partially satisfy the desired quantity of one or both of the order and/or the counter order, with any residual unsatisfied quantity left to await another suitable counter order, referred to as “resting.”
A “Clearing House,” which is typically an adjunct to the Exchange and may be an operating division thereof, is responsible for settling trading accounts, clearing trades, collecting and maintaining performance bond funds, regulating delivery, and reporting trading data to market regulators and to the market participants. An essential role of the clearing house is to mitigate credit risk via the clearing process. Clearing is the procedure through which the Clearing House becomes buyer to each seller of a futures contract, and seller to each buyer, also referred to as a “novation,” and assumes responsibility for protecting buyers and sellers from financial loss due to breach of contract, by assuring performance on each contract. A clearing member is a firm qualified to clear trades through the Clearing House.
Current financial instrument trading systems allow traders to submit orders and receive confirmations, market data, and other information electronically via a communications network. These “electronic” marketplaces, implemented by, and also referred to as, “electronic trading systems,” are an alternative trading forum to pit based trading systems whereby the traders, or their representatives, all physically stand in a designated location, i.e. a trading pit, and trade with each other via oral and visual/hand based communication.
In particular, electronic trading of financial instruments, such as futures contracts, is conducted by market participants sending orders, such as to buy or sell one or more futures contracts, in electronic form to the Exchange. These electronically submitted orders to buy and sell are then matched, if possible, by the Exchange, i.e. by the Exchange's matching engine, to execute a trade. Outstanding (unmatched, wholly unsatisfied/unfilled or partially satisfied/filled) orders are maintained in one or more data structures or databases referred to as “order books,” such orders being referred to as “resting,” and made visible, i.e., their availability for trading is advertised, to the market participants through electronic notifications/broadcasts, referred to as market data feeds. An order book is typically maintained for each product, e.g. instrument, traded on the electronic trading system and generally defines or otherwise represents the state of the market for that product, i.e. the current prices at which the market participants are willing buy or sell that product. As such, as used herein, an order book for a product may also be referred to as a market for that product.
In a futures exchange both trading and clearing may operate under a Central Counter Party (“CCP”) model, where the futures exchange functions as a counter party to each trade and to the clearing of each trade, referred to above as a novation. CCPs benefit both parties in a transaction because they bear most of the credit risk. In a scenario outside of a financial exchange, where two individuals deal with one another by themselves, the buyer bears the credit risk of the seller, and the seller bears the credit risk of the buyer. Conversely, when a CCP is used the credit risk that is held against both buyer and seller is coming from the CCP. One consequence of a CCP model is that all communication and transactions must flow through the CCP, i.e. the CCP is the SOR, and thus information and trading may only be as fast as the CCP may process it and transmit it out to the interested parties. Records are usually kept by the CCP in a database as the source of truth and communicated to other parties using messaging. The CCP's client, e.g. a clearing member, may further have its own database of at least a subset of these records and periodically, typically daily, may reconcile them with the CCP. Further, the customers of a clearing member may have their own database, necessitating similar reconciliation. This effectively serializes the distribution of data from the CCP to all interested parties and increases the latency thereof.